Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Limit proxy's memory from TiFlash's side #408

Merged
merged 6 commits into from
Jan 2, 2025
Merged

Limit proxy's memory from TiFlash's side #408

merged 6 commits into from
Jan 2, 2025

Conversation

CalvinNeo
Copy link
Member

@CalvinNeo CalvinNeo commented Dec 30, 2024 •
edited
Loading

What is changed and how it works?

Issue Number: Close tiflash#9745

What's Changed:

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Release note

Fix an issue that TiFlash may unexpectedly reject raft message when memory usage is not high

@CalvinNeo
fix
bbfb34c 
Signed-off-by: Calvin Neo <[email protected]>
@ti-chi-bot ti-chi-bot bot added the size/L label Dec 30, 2024
@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo
Copy link
Member Author

/hold for tiflash pr

@CalvinNeo
check config
5cdfdfd 
Signed-off-by: Calvin Neo <[email protected]>
@CalvinNeo
fix
533d26d 
Signed-off-by: Calvin Neo <[email protected]>
@CalvinNeo
Copy link
Member Author

/retest

@CalvinNeo CalvinNeo requested a review from JinheLin December 31, 2024 09:13
CalvinNeo
CalvinNeo commented Dec 31, 2024
View reviewed changes
proxy_components/proxy_server/src/proxy.rs
@@ -281,6 +281,18 @@ pub unsafe fn run_proxy(
.help("Set engine role label")
.takes_value(true),
)
.arg(
Arg::with_name("memory-limit-size")
Copy link
Member Author

@CalvinNeo CalvinNeo Dec 31, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or _bytes? Seems that in proxy they don't append bytes as a suffix here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "memory-limit-size" is OK. TiKV usually use "-size" for the config item in bytes
https://docs.pingcap.com/tidb/stable/tikv-configuration-file#blob-cache-size

@ti-chi-bot ti-chi-bot bot added the lgtm label Dec 31, 2024
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Dec 31, 2024

[LGTM Timeline notifier]

Timeline:

  • 2024-12-31 09:58:02.645471289 +0000 UTC m=+430818.001475832: ☑️ agreed by JinheLin.

@ti-chi-bot ti-chi-bot bot added the approved label Dec 31, 2024
@CalvinNeo
Copy link
Member Author

CalvinNeo commented Dec 31, 2024 •
edited
Loading

/hold memory limit high water ratio to 0.9

@ti-chi-bot ti-chi-bot bot removed the approved label Jan 2, 2025
JaySon-Huang
JaySon-Huang approved these changes Jan 2, 2025
View reviewed changes
Copy link
Collaborator

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Jan 2, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: JaySon-Huang, JinheLin
Once this PR has been reviewed and has the lgtm label, please assign yudongusa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@CalvinNeo
fix tests
1d70b29 
Signed-off-by: Calvin Neo <[email protected]>
@CalvinNeo
fix tests 2
4a57d45 
Signed-off-by: Calvin Neo <[email protected]>
@CalvinNeo
Copy link
Member Author

/unhold

@CalvinNeo CalvinNeo merged commit 396724d into pingcap:raftstore-proxy Jan 2, 2025
3 of 5 checks passed
@ti-chi-bot ti-chi-bot
Copy link

ti-chi-bot bot commented Jan 5, 2025

@CalvinNeo: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-unit-test 533d26d link true /test pull-unit-test

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@CalvinNeo
Copy link
Member Author

CalvinNeo commented Jan 7, 2025 •
edited
Loading

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: JaySon-Huang, JinheLin Once this PR has been reviewed and has the lgtm label, please assign yudongusa for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment Approvers can cancel approval by writing /approve cancel in a comment

This PR has changed the config.rs file, however, it is not a modification that affects the user or related to compatibilities:

  • No new configuration item is added or deleted.
  • The modified configuration is related to memory control, but is not related to any new features.
  • This changes actually fix a bug, which makes TiFlash work in an expected way.

@CalvinNeo
Copy link
Member Author

/cherry-pick raftstore-proxy-8.5

@ti-chi-bot
Copy link
Member

@CalvinNeo: new pull request created to branch raftstore-proxy-8.5: #409.

In response to this:

/cherry-pick raftstore-proxy-8.5

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot mentioned this pull request Jan 7, 2025
Merged
9 tasks
@CalvinNeo
Copy link
Member Author

/cherry-pick raftstore-proxy-8.1

@ti-chi-bot
Copy link
Member

@CalvinNeo: new pull request created to branch raftstore-proxy-8.1: #410.

In response to this:

/cherry-pick raftstore-proxy-8.1

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot ti-chi-bot mentioned this pull request Jan 7, 2025
Open
9 tasks
ti-chi-bot bot pushed a commit that referenced this pull request Jan 9, 2025
@ti-chi-bot @CalvinNeo
Limit proxy's memory from TiFlash's side (#408) (#409)
fdb7f17 
 

Signed-off-by: Calvin Neo <[email protected]>

Co-authored-by: Calvin Neo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JinheLin JinheLin JinheLin approved these changes

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

Assignees
No one assigned
Labels
lgtm size/L
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

memory_usage_limit is deduced much less then expected
4 participants
@CalvinNeo @ti-chi-bot @JaySon-Huang @JinheLin